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Deterministic  propertic  of  rounding  and  chopping  were  examined  for  both  multiplication  and  quantization 

errors. 

Statistical  properties  of  rounding  only  were  examined  for  both  multiplication  and  quantization  errors. 

Statistical  properties  examined  arc  1)  the  error  distribution  density  (d.d.),  2)  the  error  variance,  3)  the  autocorrelation 
between  successive  error  values,  and  4)  the  cross-correlation  coefficient  between  the  quantizer  input  and  the  resulting 
error.  Specific  results  were  obtained  for  zero-mean  Gaussian  random  processes. 

For  quantization  errors  the  above  properties  depend  only  on  the  ratio  of  the  process  standard  deviation  to  the 
quantization  interval  size  (o/q).  The  mapping  of  the  quantizer  input  d.d.  onto  the  quantization  error  d.d.  is  continuous. 
Consequently,  for  ol q  >  1.0,  the  error  d.d.  is  almost  exactly  uniform  between  ±q/2,  and  the  error  variance  is  very  near 

J 

q“/12.  Both  the  autocorrelation  and  the  cross-correlation  coefficients  wcie  negligible.  Furthermore,  the  equations 

show  that  the  quantization  error  approaches  arbitrarily  close  to  q“/12  as  o/q  increases,  while  the  autocorrelation  and 
cross-correlation  coefficients  approach  arbitrarily  close  to  zero. 

For  rounding  errors  the  above  properties  depend  not  only  on  o/q,  but  on  the  word  size,  N,  and  the  value  of 
the  multiplier,  J,  as  well.  Furthermore,  the  discrete  nature  of  the  computer  word  causes  a  discrete  mapping  of  the 
multiplier  input  d.d.  onto  the  multiplier; ion  error  d.d.  Consequently,  for  the  limited  range  of  parameters  considered, 
most  values  of  J  yield  an  error  d.d.  which  is  not  uniform  in  the  continuous  sense  but  shows  a  variance  approaching 

'J 

q“/12.  Similarly,  most  autocorrelation  and  c.oss-correlation  values  approach  zero,  but  stabilize  at  some  non-zero 
value  as  o/q  becomes  large.  However,  some  values  of  J  result  in  large,  non-zero  autocorerclation  and  cross-correlation 

values  and  a  variance  which  diverges  widely  from  q“/12. 
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SUMMARY 


PROBLEM 


Review  anil  extend  the  theory  of  computation  error  generation  in  digital  filters. 
Specifically  consider  fixed-point  multiplication  for  digitized  Gaussian  analog  signal  inputs. 


RESULTS 


Deterministic  properties  of  rounding  and  chopping  were  examined  for  both  mul¬ 
tiplication  and  quantization  errors. 

Statistical  properties  of  rounding  only  were  examined  i’or  both  multiplication  and 
quantization  errors.  Statistical  properties  examined  are  1)  the  error  distribution  density 
td.d.i.  2>  the  error  variance,  at  the  autocorrelation  be  twee  t  successive  error  values,  and 
4)  the  cross-correlation  coefficient  between  the  quantizer  input  and  the  resulting  error. 
Specific  results  were  obtained  for  zero-mean  Gaussian  random  processes. 

Lor  quantization  errors,  the  above  properties  d  pend  only  on  the  ratio  of  the 
process  standard  deviation  to  the  quantization  interval  size  to  q).  The  mapping  of  the 
quantizer  input  d.d.  onto  the  quantization  error  d.d.  is  continuous.  Consequently,  for 
o  q  ■  1.0.  the  eiror  d.d.  is  almost  exactly  uniform  between  *q  2.  and  the  error  variance 
> 

is  very  near  q-  12.  Both  the  autocorrelation  and  the  cross-correlation  coefficients  were 
negligible,  furthermore,  the  equations  show  that  the  quantization  error  approaches 

A 

arbitrarily  close  to  q -  I  2  as  a  q  increases,  while  the  autocorrelation  and  cross-correlation 
coefficients  approach  arbitrarily  close  to  zero. 

l  or  rounding  errors,  the  above  properties  depend  not  only  on  a  q.  but  on  the 
word  size.  N.  and  the  value  of  the  multiplier.  J.  as  well.  Furthermore,  the  discrete  nature 
of  the  computer  word  causes  a  discrete  mapping  of  the  multiplier  input  d.d.  onto  the 
multiple.,  ion  error  d.d.  Consequently,  for  the  limited  range  of  parameters  considered, 
most  values  of  .1  yield  an  error  d.d.  which  is  not  uniform  in  the  continuous  sense  but 

shows  a  variance  approaching  q-  12.  Similarly,  most  autocorrelation  and  cross¬ 
correlation  values  approach  zero,  but  stabilize  at  some  non-zero  value  as  o'\,  becomes 


Hi 


large.  However,  some  values  of  J  result  in  large,  non-zero  autocorrelation  and 
cross-correlation  values  and  a  variance  which  diverges  widely  from  q-  I  2. 


RECOMMENDATIONS 


1.  Attempt  to  derive  analytical  formulas  for  evaluation  of  the  error  variance  and 
the  cross-correlation  between  the  multiplier  input  and  the  resulting  error. 

2.  Extend  the  technique  of  analysis  to  the  problem  of  error  generation  for 
floating-point  computers. 
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INTRODUCTION 


OBJECTIVE 


We  consider  errors  that  occur  when  two  fixed-point  binary  numbers  an'  multiplied 
in  a  digital  computer.  For  example,  suppose  we  multiply  the  following  numbers  together. 
0.0 1 0 1  -»  and  1.1001  ->*.  Each  number  has  a  “word”  length  of  N  =  4  bits  plus  sign  bit. 

The  result  of  multiplication  is  a  number  2x4  =  8  bits  plus  sign  bit.  equal  to 
1.00101 101  -i  in  this  case.  To  store  the  result  in  the  computer  memory  it  is  necessary  to 

“chop”  or  “round”  it  by  discarding  the  4  least  significant  bits.  Chopping  results  when  we 
discard  these  bits,  leaving  the  first  4  bits  phis  sign  unchanged.  Rounding  is  done  by  add¬ 
ing  a  “one”  to  the  least  significant  of  the  first  4  bits  when  the  most  significant  of  the 
lower  4  bits  is  equal  to  a  “one".  Then  the  lower  4  bits  are  discarded.  If  the  most  signif¬ 
icant  of  the  lower  4  bits  is  equal  to  a  “zero"  nothing  is  added  before  the  lower  4  bits  are 
discarded.  In  the  above  example,  dropping  would  leave  the  result  I.00l0>:  rounding 

would  leave  the  result  1 .00 1 1  •>.  This  example  and  another  are  shown  in  detail  in  table  I. 

Note  that  chopping  or  round  ng  results  in  a  product  that  is  inexact.  The  difference 
between  the  cliopped  or  rounded  product  and  the  original  produet  is  an  error.  It  is  with 
this  error  and  its  charact eristics  that  we  are  concerned. 

Multiplication  errors  must  he  considered  in  evaluating  the  performance  of  digital 
filters.  A  digital  litter  is  an  algorithm  which  is  used  in  a  digital  computer  to  replace  an 
analog  filter  (reference  2).  T  he  algorithm  is  of  the  form 
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Tabic  I.  Examples  of  Rouii  ling  and  Chopping. 


EXAMPLE  NO.  1 

EXAMPLE  NO.  2 

NUMBER  A 

0.0101 

0.0101 

NUMBER  B 

1.1001 

0.1010 

A  TIMES  B 

1.00101101 

0.00110010 

RESULTS  AFTER 

rounding 

1.0011 

0.0011 

CHOPPING 

1.0010 

0.0011 

VALUE  OF  ERROR 

ROUNDING 

1.00000011 

1.00000010 

CHOPPING 

0.30001101 

1.00000010 

*> 


where  \<nT),  n  =  0.  1,2 .  is  a  sequence  of  numbers  obtained  from  the  analog 

waveform,  x(t).  This  number  sequence  is  the  input  to  the  digital  filter.  The  resulting 

output  sequence  from  the  digital  filter  is  v(nT),  n  =  0,  1.2 .  The  input  sequence. 

x(nT).  results  from  periodic  sampling  of  x(t)  at  a  rate  l's  =  1/T.  and  subsequent  conver¬ 
sion  of  these  samples  to  digital  numbers  (analog-to-digital  conversion).  Figure  1  is  a 
block  diagram  of  the  operations  needed  to  produce  a  digital  filter  which  is  equivalent  to 
the  analog  filter. 

Equation  (1)  requires  three  arithmetic  operations:  multiplication,  addition,  and 
subtraction.  Chopping  or  rounding  errors  occur  only  in  multiplication.  Over  How  errors 
occur  in  addition  and  subtraction.*  Note  also  that  the  coefficients  Kj  and  Lj  canno*  be 

specified  exactly,  in  general,  due  to  the  finite  word  length  in  the  computer.  Consequently, 
the  desired  (unquantized)  coefficient  value  differs  by  a  fixed  amount  from  the  quantized 
coefficient  value,  and  the  filter  does  not  have  the  characteristics  desired.  This  may  be  a 
problem  if  the  filter  is  sensitive  to  small  changes  in  the  value  of  the  coefficient.  Further¬ 
more.  the  recursive  equation  leads  to  dead-band  and  other  effects  (reference  2).  We  will 
not  discuss  these  effects  further  since  we  are  only  concerned  with  the  problem  of 
multiplication  error  generation. 

This  report  consists  of  three  major  sections.  The  first  section.  Quantization 
Errors,  reviews  the  theory  of  quantization  errors,  which  forms  the  foundation  of  the 
theory  of  multiplication  errors  used  until  now.  The  second  section.  Multiplication  Errors, 
defines  the  deterministic  and  statistical  properties  of  multiplication  errors  and  compares 
multiplication  errors  with  quantization  errors.  The  final  section.  Summary,  summarizes 
the  analysis  performed  in  the  second  section  and  suggests  an  approach  to  further  analysis. 


BACKGROUND 


Multiplication  errors  are  similar  to  errors  that  occur  when  an  analog  waveform 
sample  is  quantized.  It  seems  natural,  therefore,  to  extend  the  conclusions  of  the  analysis 
of  quantization  errors  to  the  analysis  of  multiplication  errors.  Quantization  errors  have 
been  analyzed  by  Bennett  (reference  3).  Widrow  (reference  4).  and  Shaver  (reference  5). 
Bennett  found  that  quantization  is  equivalent  to  adding  an  independent  (white)  noise  that 
is  uniformly  distributed  over  the  quantisation  interval  q.  to  the  original  (unquantized) 
samples.**  Me  also  determined  the  autocorrelation  of  successive  samples  of  the  quantiza¬ 
tion  errors  as  a  function  of  the  autocorrelation  of  successive  samples  of  the  analog  wave¬ 
form.  The  analog  waveform  was  Gaussian.  Widrow  determined  the  difference  between 
the  actual  variance  of  the  quantization  error  and  the  variance  of  the  uniformly  distributed 
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Figure  1.  Block  diagrams  of  an  analog  filter  and  an  equivalent  digital  filter. 


quantization  error.  Shaver  later  derived  the  cross-correlation  between  the  quantizer  input 
and  quantization  error  at  the  same  instant.  Both  analyses  were  for  a  Gaussian  analog 
waveform. 

As  we  shall  see.  chopping  and  rounding  are  exactly  the  same  operation  for  multi¬ 
plication  errors  as  for  quantization  errors.  This  has  been  noted  in  work  on  the  effects  of 
multiplication  errors  in  digital  computation.  For  example,  Knowles  and  Edwards  (refer¬ 
ences  6,  7,  8)  assumed  that  multiplication  errors  could  be  treated  as  an  additive,  independ¬ 
ent  white  noise  because  they  are  similar  to  quantization  errors.  They  analyzed  the  effects 
of  this  noise  on  the  steady-state  performance  of  digital  control  systems  of  the  form  given 
in  Equation  (1).  Asa  chcc  on  their  assumptions,  they  computed  the  autocorrelation 
functions  of  some  multiplication  error  sequences  (reference  6,  p.  2384).  A  number  of 
word  lengths,  multiplication  coefficients,  and  sampled  analog  waveforms  were  used.  They 
concluded  that  multiplication  roundoff  error  spectra  are  essentially  white  with  respect  to 
practical  sampled-data  systems.  Unfortunately,  the  paper  was  not  clear  as  to  what  kind 
of  sampled-data  system  was  used  for  the  measurements.  Also,  the  rms  value  of  the  analog 
signal  relative  to  the  quantization  interval  was  not  given. 

Gold  and  Rader  (reference  9)  also  linked  quantization  errors  with  multiplication 
errors,  referring  to  the  work  of  Bennett.  They  experimentally  verified  the  mean-square 
output  noise  for  a  one-pole  digital  filter  as  a  function  of  the  pole  position  and  word 
length  (28  and  29  bits  including  sign).  They  did  not  compute  correlation  of  successive 
output  errors,  nor  give  the  rms  value  of  the  analog  signal.  Gold  and  Rabiner  (reference 
10)  essentially  followed  Gold  and  Rader  in  their  assumptions. 

All  investigators  experimentally  verified  the  assumed  similarity  of  quantization 
errors  to  multiplication  errors  by  measuring  the  mean  square  output  noise  of  a  finite  word- 
length  digital  filter.  Only  Knowles  and  Edwards  (reference  6)  computed  correlation  of 
error  sequences,  and  did  so  for  specific  systems  and  specific  word  sizes  only. 


QUANTIZATION  ERRORS 


Later  in  this  report  we  will  compare  the  statistical  properties  of  quantization 
errors  with  those  of  multiplication  errors.  This  section  is  a  review  of  theoretical  results 
needed  for  such  a  comparison. 


QUANTIZER  CHARACTERISTICS:  DETERMINISTIC 


A  quantizer  is  used  in  an  analog-to-digitai  converter  (ADCON).  An  AIX'ON  con¬ 
verts  time  series  samples  of  an  analog  signal  to  digital  form  so  they  can  be  accepted  by  a 
digital  computer.  A  block  diagram  of  the  process  is  shown  in  figure  2.  The  block  dia¬ 
gram  does  not  represent  the  circuit  operations,  but  shows  the  equivalent  operations  in  a 
mathematical  sense.  We  will  discuss  quantization  first  and  then  cover  the  other  necessary 
aspects  of  the  ADCON. 
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Figure  2.  Block  diagram  representation  of  a  sampler  and  analog-to-digital  converter. 


In  figure  2  the  sampler  output  voltage  x*(nT)  is  a  random  variable  whose 
probability  density  function  is  continuous  in  the  interval  -B  <  x*(nT)  -S  A.  -B  and  A 
are  lower  and  upper  limits  on  the  range  of  x*(nT)  which  are  set  by  the  physical  nature 
of  the  sampler  and  quantizer. 

Quantization  is  a  process  of  subdivision  of  the  range  of  x*(nT)  into  class  intervals. 
For  rounding, 


(i  -  -  )q  <  x*(nT)  <  (i  + 
2 


(2) 


where 


-1,  0,  1 .  -  -  1,  ~ 

q  q 


(A  and  B  are  chosen  to  be  integer  multiples  of  q). 


Each  class  interval  is  of  equal  width  q.  The  quantizer  output  voltage  is  iq.  This 
is  the  center  value  of  the  class  interval.  For  example,  if  (3/2)q  <  x*(nT)  <  ( 5 / 2 )q ,  the 
quantizer  output  is  2q  volts. 

The  difference  between  the  quantizer  output  x'(nT)  and  the  input  x*(nT) 
is  the  quantization  error  e^CiiT). 

That  is, 


Cq(nT)  =  x'(nT)  -  x*(nT). 


(3) 


(Sometimes  the  error  is  defined  as  the  input  minus  the  output.  Our  convention  assumes 
that  the  quantization  error  is  added  to  the  quantizer  input.)  In  the  case  of  equation  (2) 
this  represents  a  rounding  error  since  the  input  samples  x*(nT)  are  rounded  to  the  nc  r- 
est  class  interval  center  value.  We  will  call  the  sequence  of  errors  e^tiiT).  n  =  0.  1,2 . 

quantization  noise  or  the  quantization  process.  Note  that  |e^(nT)|  *£  q/2. 

For  chopping, 


iq  <  I  x*(nT)|  <  (i  +  l)q, 


(4) 


where 


i  =  0,  1 .  A/q. 
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The  quantizer  output  voltage  is  (sgnx*)iq,  where 


(  '•  x* 
sgnx  *  =  v 


>  0 


(5) 


•I,  x*  <  0, 


and  iq  is  the  value  of  the  lower  end  of  the  elass  interval  defined  in  equation  (4).  The 
sign  of  the  quantizer  output  voltage  is  the  same  as  the  sign  of  x*(nT).  For  example,  if 
3q  <  I  x*(nT)|  <  4q  and  x*(nT)  <  0,  the  quantizer  output  is  -3q  volts.  Note  that 
|e(](nT)|  <  q. 

Quantizer  input/output  eharaeteristics  for  both  rounding  and  chopping  are  shown 
in  figure  3.  The  equivalent  ADCON  outputs  are  also  shown.  The  quantization  error  is  a 
deterministic  function  of  the  quantizer  input.  The  functions  can  be  written  as  follows: 

Rounding: 


^n(nT)l  =  -x*  +  iq. 


(6) 


where 


and 


i  =  0,  ±1,  ±2,  ... 


-q/2«e  (nT)l  <  q/2. 

q  x* 


Chopping: 


i  =  0,  < ,  2 .  x*  >  0 


e..(nT)l  =  -x*  +  iq 

q  x  * 


-q  <  e0(nT)|  <0 


i  =  0, -1, -2,  ...,  x*  <  0 


(7) 


0  <  e„(nT)|  <  q. 

q  x* 


These  functions  are  shown  in  figure  4. 

Figure  5  is  an  example  of  an  analog  signal  and  the  results  of  sampling  and  quanti¬ 
zation.  Results  for  both  rounding  and  chopping  are  shown.  A  couple  of  features  are 
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Figure  4.  Quantization  error  as  a  function  of  the  quantizer  input  voltage. 


evident.  First,  the  chopping  errors  are  generally  larger  than  the  rounding  errors.  The 
rounding-error  sequence  is  bounded  by  ±q/2  and  the  chopping  error  sequence  bounded  by 
±q.  Second,  the  chopping-error  sequence  is  quasi-periodic  and  dependent  on  the  polarity 
of  the  analog  signal  input.  The  quasi  periodicity  can  be  eliminated  by  adding  a  d.c.  level 
to  the  analog  signal.  This  results  in  a  d.c.  bias  in  the  error  sequence,  though. 

Analog-to-digital  conversion  results  in  a  binary  number,  not  an  analog  voltage. 

The  placement  of  a  binary  point  in  this  number  is  not  relevant  as  far  as  ADCON  operation 
is  concerned.  It  becomes  important  to  the  user  because  scaling  is  necessary  in  order  to 
interpret  the  ADCON  number  properly.  For  our  purpose  we  will  assume  a  leading  binary 
point.  Then,  if  the  quantizer  output  voltage  is  x'(nT)  =  iq,  where  i  =  ...  -1, 0,  +1, ...,  the 

corresponding  ADCON  output  is  just  x(nT)  =  i2~^.  There  is  a  multiplicative  relation  a 

between  x’(nT)  and  x(nT).  That  is.  x(nT)  =ax'(nT).  If  x'(nT)  =  q,  then  x(nT)  =  2~^. 

Thus,  a  =  2-^q“* .  This  scale  factor  is  included  in  figure  2.  Note  that  the  size  of  the  least 
significant  bit  at  the  ADCON  output  is  equivalent  to  q  when  referred  back  to  the  ADCON 
input. 


QUANTIZER  CHARACTERISTICS:  STATISTICAL 


We  will  consider  the  following  questions  about  the  statistical  characteristics  of  quan¬ 
tization  errors  and  quantization  error  sequences: 

1 .  What  shape  does  the  quantization  error  distribution  density  (d.d.)  have? 

2.  What  are  the  mean  and  variance  of  quantization  errors'’ 

3.  What  conditions  must  apply  for  the  error  sequence  to  be  considered  a  source  of 
white  noise? 

4.  What  conditions  must  apply  for  the  error  sequence  to  be  uncorrelated  with  the 
signal  sequence? 

Distribution  Densities  of  Quantization  Errors 


We  will  show  how  the  quantizer  input  d.d.  determines  the  quantization  error  d.d. 
For  rounding,  the  probability  that  the  quantizer  output  is  equal  to  iq  is 


P  x  4  tq  1  *  Prob  { x '  =  iq }  , 

=  Prob  j|i  -  ^  jq  <  x*  <  |i  +  Jjqj 


/*(i+l/'2)q 

Ji\- l‘2>q 


Px*(x*I  dx* 


(8) 
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where  px*(x  *)  is  the  d.d.  of  the  quantizer  input.  The  argument  of  the  random  variable 

x*  (the  sequence  index  nT)  is  omitted  since  the  sequence  is  assumed  stationary. 

The  joint  d.d.  of  the  quantizer  input  x*  and  the  quantizer  error  e*  is 


q  x 


♦(eq-x 


*)  =  P, 


)Pv*(  x*) 


=  £  5(cq  "  I'q  ■  x*i)Px*u*)- 

i=_oo 


where  —ci/2  «S  <  q/2  and  6(x)  is  a  delta  function.  We  define  6(  x)  as  a  distribution 

which  assigns  to  a  continuous  function  0(t)  the  number  #o).  That  is.  we  use  the  special 
integral  definition  (reference  1 1.  pp.  269-282) 


6<t)0(t)  dt  =0(o). 


We  assign  infinite  limits  to  the  summation  for  the  sake  of  simplicity.  In  reality  the  limits 
arc  given  by  equation  ( 2). 

The  d.d.  of  the  quantizer  error  is  then 


%'v  ■  £  %  x.<v 


x*)  dx* 


J  |  8,fq  "  <!*• 

gC  /  }i««q  ~l*q  -  x*l)px#(x*W  dx* 

I*—  I  / 


ym  _ 

a  ]CPx#,N“«q» 


HO) 

\} 


-q  rq<  q  ?2. 


For  chopping,  the  probability  that  the  quantizer  output  is  equal  to  iq  is 


/•(i+l)q 

x*  >  0 

1 

px*(  x*)  dx*. 

i  =  0.  1, 

/•iq 

x*  <  0 

•qi-Dq 

px*(x*)  dx*. 

i  =  0.  -1 

The  joint  d.d.  of  x*  and  eq  is 


Pe  ,<VX*)=  £  6Uq- liq- x*))px*(x*) 

q  x  i=Q 

x*>0 
-q<eq<0 


+  53  5,fq  '  liq  ~  x*l)px*(x-T. 

i=0 
x*<0 

(See  equation  7)  0<eq<q 

The  same  procedure  yields  the  following  d.d.  of  the  quantization  error. 


(11) 


Pe<«q>*  23  px*(iq-eq)+  Px^'V  (,3) 

q  i=0  i«0 

-q<eq<0  0<eq<q 

(We  ignore  the  terms  for  eq  *  0.  They  have  the  effect  of  introducing  a  delta  function  in 

the  d.d.  for  eq.  This  is  because  of  the  limits  on  rq  imposed  by  equations  (2)  and  (4). 

We  could  have  chosen  the  limits  so  that  the  d.d.  for  eq  is  continuous  at  eq  *  0.  but  this 

would  have  further  complicated  the  presentation.) 

Equations  (10)  and  (13)  show  that  (he  quantization  error  d.d.  results  from  a 
mapping  of  the  quantizer  input  d.d..  The  mapping  equations  are  (6)  and  (7).  The  nature 
of  the  mapping  is  illustrated  in  Inures  6  and  7.  Each  figure  shows  the  following  d.d.'s: 
quantizer  input,  quantizer  output,  quantization  error,  and  the  joint  d.d.  of  the  quantizer 
input  and  the  quantization  error.  A  zero-mean  Gaussian  d.d.  is  shown  for  the  quantizer 

•> 

input,  o/q  *  I.  where  o*  is  the  variance  of  the  d.d.. 
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marginal  and  joint  d.d.'*;  chopping. 


Example  of  marginal  and  joint  d.d.’s;  rounding 


The  error  d.d.  for  chopping  is  wider  than  that  for  rounding.  Also,  the  error  d.d. 
is  definitely  not  uniform  for  chopping,  but  is  for  rounding.  The  d.d.  for  rounding  is  actu- 
all\  not  uniform,  but  is  so  nearly  uniform  that  the  graph  scale  won't  show  the  variations.  The 
d.d.  for  chopping  will  become  more  uniform  as  o/q  is  increased.  However,  the  quantiza¬ 
tion  error  will  always  be  negatively  correlated  with  the  quantizer  input.  We  consider  only 
rounding  from  now  on,  except  when  we  look  at  multiplication  errors. 


Widrow’s  Results 


The  following  quotation  is  from  a  summary  in  reference  1 2  by  Widrow  (sec  also 
reference  4). 

The  probability  density  of  a  quantizer  output  signal  is  discrete .  consisting 
of  a  series  of  uniformly  separated  impulses  with  spacing  equal  to  the  quan¬ 
tization  box  size  q.  This  density  has  a  characteristic  function  (Fourier 
transform)  which  is  periodic  with  a  “frequency"  <t>  =  2n/q.  A  comparison 
of  quantization  with  the  addition  of  an  independent  uniformly-distributed 
(between  ±q/2)  noise  shows  that  the  quantizer  output  distribution  density 
consists  of  samples  of  the  distribution  density  of  signal  plus  noise.  Satis¬ 
faction  of  a  quantizing  theorem  ensures  that  statistics  can  be  recovered 
from  quantized  samples  and  that  quantization  noise  itself  is  precisely  flat- 
topped  distributed.  The  quantization  of  high-order  ( correlated )  signals 
compares  with  the  addition  of  first-order  noise  (statistically  independent, 
white).  When  a  multidimensional  quantizing  theorem  is  satisfied ,  quantiza¬ 
tion  noise  is  first-order  and  uncorrelated  even  though  signals  may  be  highly 
correlated. 

The  quantizing  theorem  referred  to  says  essentially  the  following:  suppose  the 
“frequency”  0  =  2zr/q  is  twice  as  high  as  the  “highest  frequency  component”  contained 
in  the  shape  of  the  quantizer  input  d.d.  px*(x*>.  It  is  then  possible  to  recover  px*(  x*) 

from  the  quantizer  output  d.d.  px-(iq). 

In  most  cases  the  theorem  is  not  completely  satisfied.  The  quantization  error  is 
then  only  approximately  uniformly  distributed  between  ±q/2.  The  quantization  error 
variance  will  be  in  error  by  some  amount.  Widrow  obtained  a  formula  for  the  error  of 
the  quantization  error.  This  was  for  a  zero-mean  Gaussian  quantizer  input.  Rounding 
only  was  treated.  The  error  is,  to  a  close  approximation, 


ev  =  2 a2  exp  (-2ff2  o2  /q2 )  ( 2  +-^ 
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The  proportion  error  is 


=  :4£exP(-2»V/r>(2*£-^)  ns) 

This  equation  is  a  monotonic  decreasing  function  of  aj q.  For  example,  the  proportion 
error  for  a/q  =  1.0  is  e  <  1.5  X  10  .  a  very  small  number.  This  means  that  the  quan¬ 
tization  error  d.d.  is  almost  uniform  for  a  low  value  of  o/q. 

Bennett's  Results 


Bennett  (reference  3  discussed  the  distortion  effect  of  sampling  and  quantization 
on  analog  waveforms.  One  of  his  results  is  the  following  (reference  3,  p.  455). 
Distortion  caused  by  quantizing  errors  produces  much  the  same  sort  of 
effects  as  an  independent  source  of  noise.  The  reason  for  this  is  that  the 
spectrum  of  the  distortion  in  the  receiving  filter  output  is  practically 
independent  of  that  of  the  signal  over  a  wide  range  of  signal  magnitudes. 

Even  when  the  signal  is  weak  so  that  only  a  few  quantizing  steps  are 
operated  there  is  usually  enough  residual  noise  on  actual  systems  to 
determine  the  quantizing  noise  and  mask  the  relation  between  it  and  the 
signal. 

Bennett  obtained  a  lengthy  formula  for  the  autocorrelation  of  the  quantization 
errors  as  a  function  of  the  autocorrelation  of  the  analog  waveform.  This  was  for  a  zero- 
mean  Gaussian  quantizer  input.  He  reduced  the  formula  to  an  accurate  approximation 
which  we  reproduce  here  (reference  3,  p.  467,  equation  (2.26)).  The  notation  is  changed 
to  conform  to  this  report. 
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and 


R  x  *  x  *(0)  ~ 

(P  vy(r)  is  defined  as  E  j  x(t)y(t  +  t)| 


and 


RXy(r) 

rxv(r)  =  — ========  ■ 

VRXX(0)Ryy(O) 

E  is  the  expectation  operator.)  Substituting  these  expressions  into  equation  (16)  we  get 
6  \ '  1  /  on  a~  \ 

ve  e  (t)  =  —  2Lj  ~  expl -4n -7r-(l -rx*x*(r))  —  (18) 

9  Q  n~  n  =  i  m~  \  q"-  / 

This  equation  is  a  monotonic  decreasing  function  of  rx*x*(r).  It  is  shown  plotted  in 

figure  8  for  a/q  =  1/3,  1/2  and  1.0.  Based  on  this  equation,  the  quantization  error 
sequence  can  be  considered  a  source  of  white  noise  for  values  of  a/q  close  to  1 .0  and  for 
quantizer  input  correlation  coefficient  values  of  0.9  or  less.  Of  course,  the  higher  o/q 
becomes,  the  higher  the  quantizer  input  correlation  coefficient  can  be  for  the  white  noise 
assumption  to  hold. 


Shaver’s  Results 


Shaver  (reference  5,  pp.  7-8)  derived  an  expression  for  the  cross-correlation 
between  the  quantizer  input  and  the  quantization  error  at  the  same  instant.  Since  the 
reference  is  not  widely  available,  we  will  reproduce  the  derivation.  (This  will  be  done  for 
zero-mean  Gaussian  processes  and  rounding.) 

The  cross-correlation  may  be  written 


(19) 
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F  ijiurc  8.  Correlation  coefficient  of  quantization  errors  vs.  correlation  coefficient 
of  successive  quantizer  input  samples. 


S’ 


Substituting  equation  (9)  into  the  above,  we  get: 


:i 


If  a  >  q.  a  good  approximation  is 


This  equation,  like  equation  (15),  decays  extremely  rapidly  for  a  >  q.  Thus,  the  quan¬ 
tization  error  sequence  is  essentially  uncorrelated  with  the  quantizer  input  sequence  for 
o  >  q. 


MULTIPLICATION  ERRORS 

MULTIPLIER  CHARACTERISTICS:  DETERMINISTIC 


The  model  we  use  for  a  multiplier  is  shown  in  figure  9.  The  multiplier  input 
sequence  x(nT),  the  multiplication  constant  J,  and  the  multiplication  output  sequence 
y(nT)  are  all  in  the  form  of  binary  words  of  length  N  bits  plus  sign.  Remember  that 
y(nT)  results  from  rounding  or  chopping  the  product  x(nT)  times  J,  which  is  2N  bits  plus 
sign.  The  error  sequence  ec(nT)  represents  the  fictitious  number  sequence  that  would  be 

added  to  the  product  x(nT)  times  J  in  order  to  produce  y(nT). 

Table  2  shows  the  rounding  and  chopping  errors  for  all  possible  combinations  of 
x(nT)  and  J.  and  a  word  length  of  N  =  2  bits  plus  sign.  Note  that  the  errors  for  J  <  0 
are  opposite  in  sign  from  the  errors  for  J  >  0.  This  occurs  in  general  for  N  >  2  as  well. 
We  will  only  consider  results  for  j  >  0  from  now  on.  More  importantly,  given  a  particu¬ 
lar  value  of  x(nT).  the  value  of  the  error  depends  on  the  value  of  the  multiplication  con¬ 
stant  J.  We  have  more  to  say  about  this  after  the  next  paragraph. 

The  number  of  values  that  the  errors  can  take  on  depends  on  the  word  length. 

The  number  of  values  is: 

Rounding:  2^  +  I 

Chopping:  2N+  *  -  1 

We  will  use  the  case  of  table  2  as  an  example.  Wc  enumerate  all  possible  errors 
in  table  3.  The  number  of  error  values#are:  rounding.  5:  chopping.  7.  Note  the  bounds 
on  the  errors.  The  rounding  errors  are  bounded  by  ±  one  half  the  least  significant  bit 
(l.s.b,)  in  the  (N  +  1  >  bit  computer  word.  Similarly,  chopping  errors  are  bounded  by 
±  one  l.s.b..  Previously,  we  showed  that  one  l.s.b.  was  equivalent  to  the  basic  quantiza¬ 
tion  interval  q  Thus,  if  we  refer  the  scale  of  the  multiplication  error  back  to  the  quan¬ 
tizer  input,  th  bounds  on  the  multiplication  error  are  the  same  as  for  the  quantization 
error.  This  is  why  the  error  values  are  shown  as  an  equivalent  voltage  referred  to  the 
quantizer  input.  This  is  the  way  we  will  show  multiplication  error  magnitudes  in  the  rest 
of  this  report.  The  value  of  N  we  use  at  any  time  is  reflected  in  the  denominator  of  the 

fractions  that  are  used.  That  is.  the  denominator  is  equal  to  2^'. 
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I'^ure  9.  Multiplier  model  showing  error  generation. 


Tabic  2.  Rounding  and  chopping  errors  for  all  possible  combinations  ot  X  and  J. 

N  =  2  bits  plus  sign. 


Table  3.  Enumeration  of  all  possible  multiplication  errors. 
N  =  2  bits  plus  sign. 


ROUNDING: 


CHOPPING: 


ERROR  VALUE 


EQUIVALENT  VOLTAGE 
BINARY  REFERRED  TO 

NUMBER  QUANTIZER  INPUT 


0.0010 

0.0001 

0.0000 

1.0001 

1.0010 


$ 


0.0011 

0.0010 

0.0001 

0.0000 

1.0001 

1.0010 

1.0011 


POSITION  OF  LEAST  SIGNIFICANT 
BIT  FOR  A  2- BIT  WORD  LENGTH 


rsj|  X) 


In  figure  4  we  showed  a  quantization  error  function  which  was  related  to  the 
quantizer  input.  It  represented  a  mapping  of  the  quantizer  input  d.d.  onto  the  quantiza¬ 
tion  error  d.d.  We  will  do  the  same  here  for  multiplication  errors.  That  is,  we  will  con¬ 
struct  mappings  of  the  multiplier  input  d.d.  onto  the  multiplication  error  d.d.  Data  of 
the  form  used  in  table  2  were  used  for  figures  10  and  1 1.  Figure  10  shows  representative 
chopping  error  patterns  for  word  lengths  of  N  =  2,  3.  and  4  bits  plus  sign.  (All  possible 
values  of  J  are  not  shown).  All  possible  values  of  the  multiplier  input  are  arranged  along 
the  horizontal  axis.  The  value  of  the  resulting  multiplication  error  is  plotted  against  the 
vertical  axis.  Each  plot  is  for  a  particular  value  of  J.  Figure  1 1  is  similar  to  10;  the 
main  difference  is  that  rounding  error  patterns  are  shown.  In  both  cases,  the  error  pat¬ 
tern  is  simply  a  mapping  of  the  multiplier  input  value  into  a  corresponding  multiplier 
error  value. 

We  compare  figures  10  and  1 1  with  figure  4.  One  basic  dissimilarity  occurs 
because  the  multiplier  input  is  a  discrete  quantity  and  the  quantizer  input  continuous.  If 
the  quantizer  input  were  discrete  (in  a  sense,  prequantized  to  a  finer  quantization  interval) 
we  would  observe  an  error  pattern  similar  to  that  for  the  multiplier.  However,  the  multi¬ 
plication  error  pattern  also  depends  on  the  value  of  the  multiplier,  J.  The  similarity 
between  quantization  and  multiplication  is  that  the  error  bounds  are  the  same:  ±q/2  for 
rounding  and  ±q  for  chopping. 

There  is  one  special  feature  illustrated  in  figure  10  which  we  will  use  in  our  sta¬ 
tistical  analysis  of  multiplication  errors.  Look  at  the  three  plots  starting  with  (N  =  2, 

1  =  1/4)  on  the  left  and  ending  with  (N  *  4,  J  =  4/16)  on  the  right.  The  plot  pattern 
for  N  *  2  is  a  basic  pattern  for  longer  word  lengths.  That  is,  the  pattern  for  x  =  0,  1/4, 
2/4,  3/4.  (N  =  2)  is  the  same  pattern  for  x  =  0,  1/8,  2/8,  3/8  (N  »  3)  and  for  x  =  4/8, 
5/8,  6/8.  7/8,  (N  =  3).  Similarly,  the  pattern  for  x  =  0.  -1/4,  -2/4,  -3/4  (N  =  2)  is  the 
same  pattern  for  x  -  0,  -1/8,  -2/8,  -3/8,  (N  =  3)  and  for  x  *  -4/8,  -5/8,  -6/ 8,  -7/8 
(N  *  3).  This  same  effect  occurs  when  we  go  to  a  word  length  of  4  bits.  The  basic  pat¬ 
tern  is  reprated  a  total  of  4  times  each  for  x  >  0  and  x  <  0.  We  conclude  from  the 
figure  that  most  error  patterns  are  based  on  basic  patterns.  For  example,  suppose 
J  *  2/16  (N  ®  4).  We  can  reduce  this  fraction  to  the  value  1/8  and  no  further.  The 
shortest  word  length  we  can  use  to  represent  this  fraction  is  N  *  3.  Thus,  the  basic  error 
pattern  is  generated  for  (J  *  1/8,  N  «  3).  And  the  error  pattern  for  (J  ■  1/16.  N  ■  4)  is 
a  copy  of  this  repeated  according  to  the  above  procedure.  The  patterns  in  figure  1 1 
show  this  same  effect. 

Figure  12  shows  an  example  of  a  multiplier  input  sequence  and  the  corresponding 
output  and  error  sequences.  This  is  similar  to  figure  5.  the  comments  made  for  figure  S 
also  apply  here  (see  p.  8).  In  addition,  note  that  the  result  of  multiplication  before 
rounding  or  chopping  wih  be  less  than  the  value  of  the  input  to  the  multiplier.  This 
result  depends  on  the  value  of  i.  The  bounds  on  the  multiplication  error  remain  the 
same.  This  is  to  be  contrasted  with  the  quantizer  where  no  operation  is  performed  on 
the  voltage  before  the  quantization  error  is  introduced. 
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I'igure  1 1.  Rounding  error  patterns. 
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DISTRIBUTION  DENSITIES  OF  MULTIPLIER  ERRORS 


Figure  13  is  an  example  of  marginal  and  joint  d.d.’s  for  a  multiplier  input  and  the 
resulting  multiplication  error.  Chopping  is  shown.  Figure  14  is  the  same  except  that 
rounding  is  shown.  The  multiplier  input  d.d.  was  derived  from  a  zero-mean  Gaussian 
input  to  a  quantizer.  Rounding  is  assumed  for  the  quantizer.  The  ratio  of  the  standard 
deviation  to  the  quantization  interval  o' q  was  4.0.  N  =  4  and  J  =  3/16.  It  is  clear  how 
the  multiplier  input  d.d.  is  mapped  onto  the  multiplier  error  d.d.  by  the  error  pattern. 
The  limits  on  the  multiplication  error  values  are  obvious  in  both  figures.  Also  note  that 
the  multiplication  error  d.d.  is  strongly  correlated  with  the  multiplier  input  d.d.  We  will 
drop  consideration  of  chopping  errors  at  this  point. 

Figure  15  shows  a  variety  of  multiplication  error  d.d.’s.  for  the  same  multiplier 
input  d.d.  as  used  for  figures  13  and  14.  Rounding  is  assumed.  In  some  cases,  the  d.d.’s 
are  approximately  uniformly  distributed.  In  other  cases  they  are  not.  The  dependence 
on  the  value  of  J  is  clear.  These  figures  illustrate  further  the  dissimilarity  between  mul¬ 
tiplication  errors  and  quantization  errors. 


ROUNDING  ERRORS:  STATISTICAL 


Model  For  Computer  Analysis 


Consider  equation  1.  If  x(nT)  is  a  zero-mean  Gaussian  random  process,  y(nT)  is 
also  a  zero-mean  Gaussian  random  process.  We  will  show  this  by  an  example  using  a 
first-order  linear  difference  equation: 

y(nT)  =  Ky(nT  -  T)  -  x(nT)  n  =  0,  1 ,  2,  ...  (27) 

The  first  value  of  x(nt)  is  x(o).  We  define  y(-T)  =  0.  Then  y(o)  =  -x(o).  Carrying  out 
the  above  equation  a  number  of  times,  we  find  that 

n 

y(nT)  =^KI1-V-x(iT)),  (28) 

i=0 

y(nT)  consists  of  a  linear,  weighted  sum  of  Gaussian  random  variables.  This  implies  that 
y(nT)  too  is  a  Gaussian  random  variable.  Also,  since 

E[  x ( nT )  1  =  0,  E[y(nT)l  =  0. 

We  can  use  this  same  procedure  for  other  forms  of  equation  1 .  All  of  this 
assumes  'hat  quantization  ana  computation  errors  are  not  present.  We  have  shown  that 
quantization  of  a  Gaussian  random  variable  results  in  a  discrete  d.d.  which  is  approximately 
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¥ure  15.  I  samples  of  mullipliealion  error  d.d.Y  N  -  4,  „/q  =  4.0.  :.nd  rounding  is  assumed. 


Gaussian  in  appearance.  We  will  assume  this  same  d.d.  for  the  input  and  output  of  a 
digital  filter.  The  output  d.d.  will  only  be  approximate  due  to  the  addition  of  computa¬ 
tional  errors.  Thus,  in  equation  (1),  the  x(nT)  and  y(nT)  are  assumed  to  be  governed  by 
the  same  d.d.  That  is,  they  are  the  result  of  quantizing  (with  rounding)  a  Gaussian 
sequence  of  random  variables.  With  these  assumptions  we  simplify  our  analysis  model  to 
that  shown  in  figure  9. 

The  analyses  described  below  have  been  programmed  and  run  on  a  CDC  1604 
computer. 


Multiplication  Error  Variance 


The  variance  of  the  computation  error  was  computed  as 


Var(ec)  =  E[(ec  -  ec)2] 

2N-1 

=  £  el  Px'(iq).  (29) 

i=-2N+l  * 


The  variance  depends  on  the  following  parameters:  J,  N,  and  a  I  q.  ec.  is  the  computation 
error  that  results  when  the  multiplier  input  is  equal  to  i2-N.  (Remember  that  the  deci¬ 
mal  form  of  the  ADCON  output  is  i2“^  when  the  quantizer  input  is  in  the  interval 
(iq  ±  q/2).)  As  we  have  shown,  the  value  of  e..  depends  on  the  value  of  J. 

i  _N 

The  variance  was  computed  for  the  following  parameters:  N  =  7;  J  =  i2-1 

(i  =  1,2 . 27  -  1);  o/q  =  2.0,  4.0,  8.0,  16.0.  Results  are  shown  in  figure  16.  The 

variance  for  a  uniformly  distributed  error  is  shown  as  a  horizontal  line  in  the  center  of 
the  graphs.  The  interval  of  ±  10  percent  of  this  value  is  also  shown. 

The  first  graph  is  for  a/q  =  2.0.  Except  for  a  small  range  of  values  for 
J  -  32/128,  64/128,  and  96/128  most  of  the  values  of  error  variance  fall  outside  the  10  per¬ 
cent  interval.  For  comparison,  the  proportional  error  in  the  variance  of  the  quantizing  error 

for  o/q  =  1.0  is  about  one  part  in  107.  So,  we  see  that  the  performance  of  the  compu¬ 
tation  error  variance  is  very  much  worse  than  that  of  the  quantizing  error  variance.  As 
o/q  is  increased,  the  computation  error  variance  converges  to  the  value  of  a  uniformly 
istributed  error  for  most  values  of  J.  For  certain  values  of  J  the  error  variance  con¬ 
verges  to  some  other  value.  (See  values  for  J  =  16/128,  32/188 .  112/128.) 

The  data  points  in  figure  16  are  shown  connected  for  ease  in  visualization.  This 
does  not  imply  that  we  are  safe  in  interpolating  variance  values  for  N  >  7  and  values  of  J 
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that  arc  representable  only  for  N  >  7.  For  example,  we  could  have  plotted  variances  for 
N  =  6.  Then,  the  two  downward  spikes  at  J  =  43/128  and  J  =  85/128  would  not  have 
been  indicated. 

There  is  an  upper  limit  to  the  standard  deviation  allowable  for  the  input  to  a  mul¬ 
tiplier.  This  limit  is  set  by  the  size  of  N,  and  is  imposed  at  two  points.  The  first 

is  the  input  to  the  quantizer.  If  the  quantizer  input  exceeds  (2^  -  l/2)q  volts  overload¬ 
ing  occurs.  The  second  point  is  the  output  of  the  digital  filter  represented  by  equation 
( 1 ).  It  is  possible  for  the  sum  of  all  the  terms  to  be  too  big  for  the  word  size  (>  1 .0  in 
this  case).  (That  is.  overflow  occurs.)  In  the  practical  case,  we  can  limit  the  standard 
deviation  so  that  quantizer  overloading  or  register  overflow  will  occur  infrequently.  A 
good  value  for  the  standard  deviation  is  one-fourth  the  voltage  which  is  equivalent  to  the 
maximum  size  of  the  word.  The  probability  of  overloading  in  either  the  positive  or 

negative  sense  is  then  about  6.33  X  10~^.  In  figure  16.  the  maximum  value  of  o/q  that 
can  be  accommodated  for  N  =  4  is  a/q  =  4.0.  Other  maximum  values  are  N  =  5, 
o/q  =  8.0;  N  =  6,  o/q  =  16.0;  N  =  7.  o/q  =  32.0;  etc.  We  call  these  limits  on  o/q  the  4o 
load  limits. 


Autocorrelation  Computations 


The  autocorrelation  coefficient  for  two  successive  computation  errors  is 
E  |fe(n  |T)ec(  n  VT)| 


re  e  (T)  =  ,  , 

ecec  E  |e£(  n  ,T)| 


^N_,  2N-1 


-M  .  .  _M  .  1  J 


_  j=-2N+i  j=-2N+l 


2N-1 


1  =  Ini  -  n -il 


(30) 


Z)  4Px'(k*> 

Kl  1 


i=-2N+l 


where 


lYxMiq.jq>  =  Prob|x'(ii|T)  =  iq,  x'(nVT)  =  jq| 

=  Prob|(i  - -!;)q  <  x*(njT)  <  (i  +  ^)q.  (j  -  ^)q  <  x*(nnT)  <  (j  + 
p(i+l/2)q  /*< j+l/2)q 


A  i+l /2)q  /*(  j+l/2)( 
i-l'2)q  y<j-l/2)q 


Px*x*(  x*(njT).  x*(n2T))dx*(n|T)  dx^i^T). 


and  Px*  x*(  x*(ii|T),  x*(n->T))  is  a  bivariate  Gaussian  d.d.  with  autocorrelation 

rx*x*(T).  Px- x'(iq, jq) was  obtained  numerically  using  Simpson's  rule.  (Note  that  the 

results  described  are  for  a  given  autocorrelation  value  of  the  quantizer  input,  nut  the  mul¬ 
tiplier  input.) 

The  autocorrelation  coefficient  was  computed  for  the  following  parameters: 

N  =  6:  J  =  i 2-^  (i  =  1.2 . 2^  -  1 ):  a  q  =  I.  2 . 6:  rx»x*(T)  =  ().*■>.  r .  e  (t»  is 

plotted  as  a  function  of  J  in  figure  17.  Here  too.  the  data  points  are  shown  connected 
for  visual  effect  only.  (Results  for  o/q  >  6.0  were  not  obtained  due  to  the  amount  of 
computer  time  needed. 

The  values  are  also  much  higher  than  the  value  of  the  quantizing  error  correlation 

coefficient  for  o/q  =  1.0  (see  figure  8:  when  rv  *  *  *  =  0.9.  r,  ,  =  .01 17).  In  figure  I7. 

*  A  sqeq 

for  o/q  =  6.0,  only  two  values  of  J  (13/64  and  51  64)  yield  a  lower  value.  Other  values 
of  J  yield  lower  values  for  o/q  <  6.0.  But  the  computation  error  correlation  coefficient 
later  comes  back  up  when  o/q  =  6.0.  The  coefficient  values  seem  to  stabilize  for 
J  =  16/64,  32/64  and  48/64  for  even  this  restricted  range  of  o  q.  Unfortunately,  since 
we  do  not  have  data  for  o/q  >  6.0.  we  can  only  speculate  that  the  correlation  coefficient 
will  be  low  enough  for  most  cases  of  interest.  It  is  not  low  enough,  for  the  most  part, 
when  o/q  <  4.0.  nor  for  N  <  4  when  the  4o  load  limits  are  taken  into  account. 


Cross-Correlation  Coefficient 


The  cross-correlation  coefficient  between  the  multiplier  input  xtnT)  and  the 
resulting  multiplier  error  ec(nT)  at  the  same  instant  is 

recx<°>  *  r€cx*(°> 


E  {  x‘(  nT)ec(  nT)} 

Ve  J I  x’(  nT>)  -  i  E|(ec(nT)|:| 


(32) 
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The  cross-correlation  coefficient  was  computed  for  the  following  parameters: 

N  =  6:  J  =  i2~‘V<  (i  =  1.2 .  2^  -  1 ):  a! q  =1.2 .  30.  Its  behavior  as  a  function  of 

o/q  is  evident  in  figure  18. 

Each  curve  tends  to  vary  wildly  starting  with  o/q  =  1.0.  Then,  it  settles  down  to 
some  non-zero  positive  value.  The  reason  for  the  positive  value  can  be  traced  to  the 
appearance  of  the  error  patterns  in  figure  1 1.  When  x  >  0.  the  mapping  onto  ec  occurs 

for  mere  positive  values  of  e  than  negative  values.  When  x  <  0.  the  mapping  occurs  for 

more  negative  values  of  ec  than  for  positive  values.  Each  curve  settles  more  quickly  for 

those  values  of  J  that  are  representable  for  N  <  6.  J  =  32  64  is  the  most  extreme 
example.  Next  comes  J  =  16/64  and  48,  64.  Then.  J  =  8  64.  24  64.  40  64  and  56  64. 
and  so  on.  Values  of  J  representable  only  for  N  >  6  seem  to  have  the  least  tendency  to 
settle  down  in  the  range  shown  for  o/q.  This  behavior  is  correlated  with  the  behavior  of 
the  multiplication  error  variance  as  a  function  of  o/q. 

The  rate  of  settling  down  seems  to  be  inversely  correlated  with  the  final  non-zero 
value  of  the  cross-correlation  coefficient.  That  is.  the  faster  it  settles  down,  the  farther 
from  zero  it  stays  as  o/q  becomes  large. 

Generalization  of  these  results  for  N  >  6  is  not  always  safe.  Figure  ll)  shows 
why.  It  is  a  plot  of  the  cross-correlation  coefficient  as  a  function  of  J  for  o/q  =  30.0. 
The  plot  is  in  two  paits.  The  top  part  is  for  values  of  J  that  are  representable  for  both 
N  =  6  and  7.  The  bottom  part  is  for  only  those  values  of  J  that  are  representable  for 
N  =  7.  The  top  is  uniform  in  appearance.  But.  it  is  obvious  that  pitfalls  occur  if  we  try 
to  extrapolate  performance  for  N  >  6  for  numbers  that  are  only  representable  for  N  >  6. 
The  bottom  pail  shows  additional  variations  that  are  not  nrcdictable  by  looking  at  the 
top  part.  They  are  also  significantly  non-zero  in  some  cases  when  we  consider 
that  o/q  =  30.0  is  very  close  to  the  4o  load  limit  for  n  =  7.  Incidentally,  the  spikes  that 
occur  for  J  =  43/128  and  J  =  85/128  are  at  the  same  position  as  the  downward  spikes  in 
figure  1 6  for  o/q  =  1 6.0. 
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SUMMARY 


We  have  examined  the  following  statistical  properties  of  quantization  and 
multiplication  rounding  errors: 

1.  d.d.  of  the  error, 

2.  variance  of  the  error, 

3.  autocorrelation  coefficient  between  successive  error  values,  and 

4.  cross-correlation  coefficient  between  the  quantizer  or  multiplier  input  with 
the  resulting  errors. 

Specific  results  were  obtained  for  zero-mean  Gaussian  random  processes  as  follows. 


QUANTIZATION  ERRORS 


The  statistics  of  quantization  errors  depend  only  on  the  ratio  of  the  process 
standard  deviation  to  the  quantization  interval  size  (cq).  The  mapping  of  the  quantizer 
input  d.d.  onto  the  quantization  error  d.d.  is  continuous.  Consequently,  for  o/q  >  1.0. 
the  error  d.d.  is  almost  exactly  uniform  between  ±q/2.  and  the  error  variance  is  very  near 

q^/12.  Both  the  autocorrelation  and  the  cross-correlation  coefficients  were  negligible. 
Furthermore,  the  equations  show  that  the  quantization  error  approaches  arbitrarily  close 

to  q~/12  as  o/q  increases,  while  the  autocorrelation  and  cross-correlation  coefficients 
approach  arbitrarily  close  to  zero. 


MULTIPLICATION  ROUNDING  ERRORS 


For  rounding  errors,  the  above  properties  depend  not  only  on  o/q.  but  on  the 
word  size,  N,  and  the  value  of  the  multiplier,  J,  as  well.  Furthermore,  the  discrete  nature 
of  the  computer  word  causes  a  discrete  mapping  of  the  multiplier  input  d.d.  onto  the 
multiplication  error  d.d.  Consequently,  for  the  limited  range  of  parameters  considered, 
most  values  of  J  yield  an  error  d.d.  which  is  not  uniform  in  the  continuous  sense  but 

shows  a  variance  approaching  q~/l  2.  Similarly,  most  autocorrelation  and  cross¬ 
correlation  values  approach  zero,  but  stabilize  at  some  non-zero  value  as  a! q  becomes 
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large.  However,  some  values  of  J  result  in  large,  non-zero  autocorrelation  and 
cross-correlation  values  and  a  variance  which  diverges  widely  from  q-/12. 


Shortcomings  of  Present  Analysis 


It  should  be  remembered  that  the  results  are  based  on  data  obtained  for  digital 
words  of  size  N  <  7  bits  plus  sign  bit.  We  have  not  considered  the  possibility  of  round¬ 
ing  off  fewer  bits  leaving  a  result  of  multiplication  which  is  greater  in  size  than  the  multi¬ 
plier  input  word  size.  Neither  have  we  obtained  data  for  word  sizes  greater  than  8  bits. 
Operation  with  word  sizes  greater  than  8  bits  is  of  interest  due  to  the  increased  availa¬ 
bility  of  process  control  computers  in  the  12-  and  16-bit  word-size  range.  (11  bits  plus 
sign  and  15  bits  plus  sign.) 


An  Approach  to  Further  Analysis 


Since  generalization  of  the  results  of  this  paper  to  word  sizes  greater  than  8  bits 
has  its  shortcomings,  we  suggest  the  following  approach.  First,  assume  ideal  multiplica¬ 
tion  error  statistics  (shape  of  d.d.,  variance  size,  and  auto-  and  cross-correlation  coeffi¬ 
cients)  for  the  analysis  of  the  effect  of  multiplication  errors.  Analysis  approaches  are 
worked  out  in  reference  2.  Once  the  preliminary  design  is  fixed,  perform  a  Monte  Carlo 
simulation  of  the  digital  filter  in  a  digital  computer.  Then  compute  the  error  d.d.,  the 
error  variance,  and  auto-  and  cross-correlation  coefficients  for  each  multiplier  coefficient 
in  the  filter.  Do  this  for  a  representative  set  of  filter  input  sequences.  (Sequences  of 
correlated  Gaussian  random  variables  are  easily  generated  using  computer  programs.)  If 
the  results  for  a  multiplier  coefficient  are  bad,  it  may  be  possible  to  get  good  results  by 
using  a  siightly  different  coefficient  value.  Of  course,  the  change  in  filter  characteristics 
would  have  to  be  acceptable. 
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LIST  OF  SYMBOLS 

A„~B:  upper  and  lower  limits  (volts)  of  the  sampler  and  quantizer 

a:  a  multiplicative  relation  between  x'(nT)  and  x(nT) 

6(x):  the  delta  function 

ec(nT):  multiplication  error  sequence 

ec.(nT):  the  multiplication  error  that  results  when  the  multiplier  input  is  equal  to 

’  i2-N 

Cq(nT):  quantization  error  sequence 

ev :  error  of  the  variance  of  the  quantization  error 

e'v\  proportional  error  of  the  variance  of  the  quantization  error 

fs:  sampling  frequency 

Kj,L,:  digital  filter  coefficients. 

N:  the  number  of  bits  in  a  digital  word  (excluding  the  sign  bit) 

0:  “frequency” 

0(t):  any  continuous  function 

q:  the  size  of  the  basic  quantization  interval  (volts) 

RXy(r):  the  cross-correlation  between  any  two  random  variables  x  and  y:  the  auto¬ 

correlation  of  x  when  y  =  x 

rxy(r):  the  cross-correlation  coefficient  between  any  two  random  variables  x  and  y: 

the  autocorrelation  coefficient  of  x  when  y  =  x 
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~) 

a~:  the  variance  of  a  (Gaussian)  d.d. 

T:  the  time  interval  between  analog  waveform  samples  and  numbers  of  a 

number  sequence 

x(t):  analog  waveform 

x*(nT):  analog  waveform  sampler  output  sequence 

x(nT):  ADCON  output  sequence:  the  input  sequence  to  a  digital  filter 

x'(nT):  quantizer  output  sequence 

y(n.T):  digital  filter  output  sequence 

SPECIAL  TERMS 

ADCON:  analog-to-digital  converter 

d.d.:  (probability)  distribution  density 

l.s.b. :  least  significant  bit 
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